Home | | Latest | About | Random
# 13 Matrix-vector equation and linear system of equations revisited.
From what we saw in [[1 teaching/smc-spring-2024-math-13/linear-algebra-notes/12-matrix-vector-product|notes 12]], we notice that if $A$ is some $n\times k$ matrix and $\vec x$ is a $k\times 1$ vector with entries some variables, $\vec x = \begin{bmatrix}x_{1}\\x_{2}\\\vdots\\x_{k}\end{bmatrix}$, then the matrix-vector product $A\vec x$ looks like the "left-hand-side" of a linear system of equations. And indeed, we can translate back-and-forth between a system of linear equations with a **matrix-vector equation**.
> **The following two problems are equivalent:**
> (1) Solve for $x_{1},x_{2},\ldots,x_{k}$ where $$\left\{\begin{array}{}a_{11} x_{1} + a_{12} x_{2} +\cdots + a_{1k}x_{k} = b_{1} \\ a_{21} x_{1} + a_{22} x_{2} +\cdots + a_{2k}x_{k} = b_{2} \\ \vdots \\a_{n1} x_{1} + a_{n2} x_{2} +\cdots + a_{nk}x_{k} = b_{n} \\ \end{array} \right.$$(2) Solve for the vector $\vec x = \begin{bmatrix} x_{1}\\x_{2}\\\vdots\\x_{k}\end{bmatrix}$ in the matrix-vector equation $A\vec x = \vec b$, where $$
\underbrace{\begin{bmatrix}
a_{11} & a_{12} & \cdots & a_{1k} \\
a_{21} & a_{22} & \cdots & a_{2k} \\
\vdots \\
a_{n1} & a_{n2} & \cdots & a_{nk}
\end{bmatrix}}_{A}\underbrace{\begin{bmatrix}
x_{1} \\
x_{2} \\
\vdots \\
x_{k}
\end{bmatrix}}_{\vec x}=\underbrace{\begin{bmatrix}
b_{1} \\
b_{2} \\
\vdots \\
b_{n}
\end{bmatrix}}_{\vec b}
$$
**Remark.** Since these problems are equivalent, we sometimes just call the matrix-vector equation $A\vec x=\vec b$ a **linear system**, and generally there will be no confusion about it.
**Example.**
Solve for a column vector $\vec x$ (of appropriate size) such that $A\vec x =\vec b$, where $A = \begin{bmatrix} 1 & 2 \\ 5 & 7\end{bmatrix}$ and $\vec b = \begin{bmatrix}3 \\ 1\end{bmatrix}$. That is, solve $$
\begin{bmatrix}
1 & 2 \\
5 & 7
\end{bmatrix} \vec x = \begin{bmatrix}
3 \\
1
\end{bmatrix}
$$ for some appropriate column vector $\vec x$.
Ok, since $\vec x$ is a column vector, it must be $2\times 1$ for this to make sense, so $\vec x = \begin{bmatrix}x_{1}\\x_{2}\end{bmatrix}$ for some unknown $x_{1},x_{2}$. This is equivalent to the system of linear equation $$
\left\{
\begin{array}{}
x_{1} & + & 2x_{2} & = 3 \\
5x_{1} & + & 7x_{2} & = 1
\end{array}
\right.\quad ,
$$which we know how to solve. We write down the augmented matrix and row reduce to an EF: $$
\begin{array}{}
x_{1} \ \ x_{2}\quad\ \ \ \ \ \ \\
\begin{bmatrix}
1 & 2 & \vdots & 3 \\
5 & 7 & \vdots & 1
\end{bmatrix} & \stackrel{\text{row}}\sim & \begin{bmatrix}
1 & 2 & \vdots & 3 \\
0 & -3 & \vdots & -14
\end{bmatrix}
\end{array}
$$so we have $x_{2} = \frac{14}{3}$ and $x_{1} = 3 - 2x_{2} = 3- \frac{28}{3} = -\frac{19}{3}$. Hence we have $$
\vec x = \begin{bmatrix}
- 19 / 3 \\
14 / 3
\end{bmatrix}.
$$
Here we make a useful observation:
> **Solving matrix-vector equation.**
> If we have a matrix-vector equation $A \vec x = \vec b$ for some unknown vector $\vec x$, then this is equivalent to solving a linear system with augmented matrix $$\left[A \ \ \vdots \ \ \ \vec b \right]$$ which is the matrix $A$ with the column vector $\vec b$ augmented to the right.
But also, let us make another important observation. Since we noted that the expression $A\vec x$ is some linear combination of the columns of $A$, the question $A\vec x = \vec b$ is really the following:
> **Linear combination interpretation of matrix-vector equation.**
> The question of solving for $\vec x$ such that $A\vec x=\vec b$ for some given $A,\vec b$ is the same as asking: Is there a linear combination of the columns of $A$ that make up $\vec b$? Or, is the vector $\vec b$ some linear combination of the columns of $A$? If so, what is this combination, and what are all the combinations?
So in above example of solving $$
\begin{bmatrix}
1 & 2 \\
5 & 7
\end{bmatrix} \vec x = \begin{bmatrix}
3 \\
1
\end{bmatrix}
$$the fact that it is consistent and we got $\vec x = \begin{bmatrix}- 19 / 3 \\14 / 3\end{bmatrix}$ means the vector $\begin{bmatrix}3\\1\end{bmatrix}$ can be made as a linear combination of the columns $\begin{bmatrix} 1\\5\end{bmatrix}$ and $\begin{bmatrix} 2\\7\end{bmatrix}$, and the weights are $-\frac{19}{3}$ and $\frac{14}{3}$ respectively. That is, $$
-\frac{19}{9} \begin{bmatrix} 1\\5\end{bmatrix} + \frac{14}{3} \begin{bmatrix} 2\\7\end{bmatrix}= \begin{bmatrix} 3\\1\end{bmatrix}.
$$
Take a moment to digest all these equivalent connections!!
We can now make the following theorem:
> **Theorem** The linear system $A\vec x = \vec b$ is consistent if and only if $\vec b$ is a linear combination of the columns of $A$.
One thing to keep in mind: Since the matrix-vector product $A\vec x$ gives a column-vector that is a linear combination of columns of $A$, and that $AB$ is a matrix where the $i$-th column $AB$ is just $A\vec b_{i}$, where $\vec b_{i}$ is the $i$-th column of $B$, this means that
> $AB$ is just a matrix where each column is a linear combination of columns of $A$.
I invite you to think deeply about it.